feat(pbs): dial builders outside the config for ePBS requests by JasonVranek · Pull Request #506 · Commit-Boost/commit-boost-client

JasonVranek · 2026-10-07T04:05:50Z

Builds on #505. There, a bid or preferences request whose auth data matches no relay entry gets 400, so a validator key whose builder config names a builder the operator has not added to Commit-Boost gets no bids from it. This PR has Commit-Boost dial the builder the auth data names, behind an SSRF guard, and routes auth data written as a builder URL. It also answers the signed block with 202 once it has been forwarded, whatever the builders answer, which addresses a review comment on #505.

What it does

URL-form auth data. Besides the hostname (the builder-specs default), auth data that is an http or https URL matches the first relay entry with the same scheme, host and port. The path and the pubkey in the relay URL do not count.
Builder parameters. Either form may end in ? and form-encoded parameters for the builder, such as builder-a.example.com?filter=1, a way for a proposer to state a preference like transaction filtering. Commit-Boost routes on the part before ? and forwards the signed auth, parameters included, unchanged, so the parameters arrive signed by the proposer. A builder that does not accept them answers 400, so a proposer sets them only for builders that accept them. Without a ?, routing is unchanged.
Dialing builders outside the config. A bid or preferences request whose auth data matches no relay entry goes to the builder it names: https://<hostname> on the default port, or the URL's scheme, host and port. There is no setting to turn it off. Dialed requests count under relay_id="dial", and each dial logs the builder's origin and the addresses it connects to. A key without builder config sends Commit-Boost's own hostname, so Commit-Boost dials https://<its hostname> on port 443: 400 when that hostname resolves to a loopback or private address, as it usually does, and otherwise one request to whatever serves HTTPS on that host, which is Commit-Boost only behind a TLS proxy, where the loop guard below stops it.
The signed block gets 202 once it has been forwarded, whatever the builders answer. This answers a review comment on epbs builder api endpoints #505: relays may do nothing with these requests, so an apparent failure should not be reported as one, and the beacon node gossips the block anyway. The block still goes only to configured relays, since Commit-Boost keeps no auction state; a dialed builder gets it over gossip.
A test for epbs builder api endpoints #505's JSON passthrough. epbs builder api endpoints #505 returns the relay's bid body unchanged when the beacon node asks for the encoding the relay answered in. A new mock relay option serves its JSON bid pretty-printed, and a test checks that the beacon node gets exactly those bytes.

The ePBS docs page gets a section on builders outside the config, which also says what a key without builder config gets, the URL form and ? parameters in the routing reference, 500 in the metrics table for preferences only, and troubleshooting rows for both 400s and for that warning.

Scope

Commit-Boost now makes outbound requests to hosts named in request data. The proposer signs that data, but Commit-Boost does not verify the signature, so the target is treated as untrusted, and anyone who can reach Commit-Boost's PBS port can choose it:

The host is resolved once, after the timing headers and the deadline are checked; an IP address needs no lookup. The lookup and the dial share the bid's remaining time (timeout_register_validator_ms for preferences). If the name does not resolve, the answer is 400 and nothing is dialed. A lookup or dial setup that runs out of time counts as a builder that did not answer: no bid (204), or 500 for preferences.
At most 32 dial lookups run at once, and further requests get the same "did not answer" result. A lookup that times out keeps its blocking thread until the resolver returns, and configured relays' own lookups share that pool, so without the cap, requests naming slow-resolving hosts could stall the relays' DNS.
The dial connects only to the checked addresses (reqwest resolve_to_addrs), ignores HTTP(S)_PROXY and ALL_PROXY (a proxy would resolve the host again), follows no redirects, and sends none of the operator's relay headers.
A request carrying X-CommitBoost-Version, which every Commit-Boost dial sends, never leads to a dial, so a dial that reaches a Commit-Boost, this one included, goes no further. Configured relays still serve such a request, so chained Commit-Boosts keep working.
Auth data comes from the request, and a dialed builder's response comes from a host the request chose, so Commit-Boost logs both escaped, keeps a logged error body to 1 KiB, and logs a dial target as its origin only, without any user:password@.
A public address of the host Commit-Boost runs on is not recognized as its own. A dial to it reaches services listening on all interfaces, as a POST of an SSZ body to a fixed builder-API path.
Each dial builds its own HTTP client, because the address pin is per client, so it pays a client build and a fresh connection. Configured relays are unaffected.

Testing

cargo test --all-features: 402 passed (392 on part 1). Unit: URL-form auth data decoding (only http and https; opaque data such as builder-a:prod and host:port parse as URLs and are refused), the dial target for each auth data form (URL, hostname, IPv6 hostname, a hostname with ? parameters, an internal address, a non-hostname, a non-HTTP scheme), every resolved address checked, the lookup cap refusing a hostname but not an IP address, the dial pinned to the given address with redirects refused, the blocked-address table, and URL-form routing to relay entries and routing on the part before ?. End to end: a bid dialed with its auth, ? parameters included, unchanged, a bid naming Commit-Boost's own URL dialed once and answered with the second hop's 400, a request from a Commit-Boost not dialed but still served by a configured relay, preferences dialed, an unmatched localhost refused as loopback, and no lookup once the deadline has passed. A builder's 400 and 401 reach the beacon node even with an error body over the 1 KiB read cap. The signed block gets 202 when every builder rejects it, and a pretty-printed JSON bid reaches the beacon node byte for byte.
Reverting each change fails a test: dropping the Commit-Boost version header from dialed requests fails the dial-to-self test, which then loops until the deadline and answers 204, putting the 500 for a block no builder accepted back fails the all-reject test, and removing epbs builder api endpoints #505's passthrough arm fails only the passthrough test.
Not covered by a test: the proxy bypass (the test would set HTTP_PROXY in a process that runs other tests in parallel), the escaping in logs and the 1 KiB cap on a logged error body, the refusal when the dial setup leaves no time, and the lookup's timing: its timeout and the time it leaves the dial (a local lookup finishes inside tokio's 1 ms timer tick, so a name slow enough to matter needs the network).
Kurtosis, on the full stack at part 4's tip, which carries this PR's code: the cb-testing modes pass, and so does the live bid-stream gate against Helix. In the block-submission mode, the signed block was forwarded 17 times and buildoor accepted all 17. Dialing ran on a devnet too, with a test-only build that turns off the private-address check (cb-testing --assert dial; Docker addresses are all private). Each key's auth data was http://buildoor:8080, which no relay entry matches, so Commit-Boost dialed buildoor 95 times, for bid and preferences requests. All 17 observed slots were built from those bids, no dial was refused, and none looped.

Pin the lighthouse crates to sigp/lighthouse unstable at 31d8cfd, whose gloas containers hash as EIP-7495 progressive containers, matching the consensus specs. Mirror lighthouse's own [patch.crates-io] entries for the progressive ssz stack (a consumer does not inherit a dependency's patches) and pin the blstrs_plus patch to an explicit rev. lighthouse dropped `TestRandom` for an `arbitrary` generator, so `TestRandomSeed` now draws from `arbitrary::Arbitrary`. Validated BLS points do not randomize under `arbitrary`, so tests that need a real key use `BlsSecretKey::random()`. Also adapt to `ForkName::Heze` and to `ExecutionRequests` no longer implementing `ssz::Decode`.

Tests discovered a free port, dropped the listener and let the server rebind it, leaving a window where another process could take the port. PbsService and SigningService gain `run_with_listener`, the mock SSV servers take a bound listener, and the legacy suites hand their listeners straight to the server. Also add `wait_for_ready`, which polls /status instead of sleeping a fixed 100 ms.

From the Gloas fork the beacon node calls three builder-API endpoints on Commit-Boost instead of get_header and get_payload: - POST /eth/v1/builder/execution_payload_bid/{slot}/{parent_hash}/{parent_root}/{proposer_pubkey} sends the request to the one relay its auth data names and returns that relay's bid (200), its 400 or 401, or 204 when it has no bid or fails. Commit-Boost does not validate or rank the bid: the beacon node checks it, and each request names exactly one builder. - POST /eth/v1/builder/builder_preferences/{proposer_pubkey} forwards to the relay the auth data names and returns that builder's answer. - POST /eth/v1/builder/beacon_blocks forwards the SSZ block bytes unparsed to every configured relay and answers 202 when one accepts. Requests must carry Eth-Consensus-Version: gloas. A body without Content-Type is JSON and a request without Accept gets JSON, as builder-specs requires. Bid requests need Date-Milliseconds and X-Timeout-Ms; Commit-Boost clamps the deadline to one slot, keeps proposer_deadline_buffer_ms back for the return trip and sends the rest to the builder as its own X-Timeout-Ms, or returns 204 when nothing is left. Routing follows builder-specs #168: auth data matches a relay when it equals the lowercase hostname of the relay URL. The first match in config order wins and unmatched auth data is a 400. Commit-Boost never verifies auth data; the builder does. Relay-supplied amounts are saturated. The new operator page (docs/get_started/epbs.md) covers writing each validator key's builder config through the keymanager API, with a copy-paste call, plus routing, timing, metrics and troubleshooting. Unreleased, shipping from v0.12.0-rc1.

…links to

condition and lint test

Auth data that is an http(s) URL now also routes to the relay entry with the same scheme, host and port. Either form may end in `?` and parameters for the builder: Commit-Boost routes on the part before `?` and forwards the signed auth, parameters included, unchanged. When no relay entry matches, Commit-Boost dials the builder the auth data names for a bid or preferences request: `https://<hostname>`, or the URL. The target is resolved once, and the lookup and the dial share the request's budget. A name that does not resolve, or any address that is not public unicast, gets 400; a lookup or dial setup that runs out of time counts as a builder that did not answer. At most 32 lookups run at once, since a timed-out lookup keeps its blocking thread. The dial goes only to the checked addresses, with no proxy and no redirects. A request from a Commit-Boost never leads to a dial, so a dial that reaches a Commit-Boost, this one included, goes no further. Auth data, a dialed builder's error body and a dial target come from the request, so they are logged escaped, the body capped at 1 KiB and the target as its origin. The signed block gets 202 once it has been forwarded, whatever the builders answer, since the beacon node gossips it anyway. A block whose bid came from a dialed builder, which is not a relay entry, would otherwise always get 500. When no builder accepts the block, Commit-Boost logs a warning.

proposer_deadline_buffer_ms must now be greater than 0 as well as less than one slot. With 0 the builder gets the whole deadline, so a bid it sends at the deadline reaches the beacon node late. The range is one check in PbsConfig::validate, so the service, a hot reload and a custom PBS module's loader all refuse it. The test fixtures use the default, 50. The metrics catalog documents the ePBS endpoint values, the codes the ePBS endpoints return to the beacon node, the relay_id="dial" label, and that ePBS bids set the relay gauges. The ePBS page corrects two builder config bullets: a top-level min_bid does not floor an entry that sets its own, and the per-client max_execution_payment defaults are dropped. A builder Commit-Boost refuses to dial is not counted under relay_id="dial".

A client per dial existed only because resolve_to_addrs pins addresses per client. The shared client's resolver now checks every address it resolves, so a dial still connects only to checked addresses, and a second dial to a builder reuses the first one's connection instead of a new TCP and TLS handshake. Dial hosts are chosen by the request, so the client keeps at most one idle connection per host, for one slot. The check before the dial is unchanged, so a blocked target still gets 400 before anything is sent.

decode_auth_data_url only ever gets the auth data address, the part before any `?`, so its parameter and doc now say so. dial_target's comparison of the parsed host with the address passes only a hostname already in the builder-specs default form, since parsing also accepts a port, userinfo, a path, upper case and numeric IPv4.

A dial to a hostname looked the host up twice in a row on the request path: once in dial_relay's check, and again in DialResolver when the connection opened. The resolver now takes the check's addresses while they are under 2 s old, checking them again, and looks the host up itself otherwise, so a later connection still sees a DNS change. The addresses are kept with port 0, as the resolver's own lookup returns them, so a dial to the same host on another port connects to its own.

Routing: a request whose auth data names a builder looks first among the relays serving the key, then among [[relays]] in config order, before dialing. A key in a mux whose auth data named a relay configured only in [[relays]] was dialed at https://<host>:443 without that entry's port or headers, or, on a private address, refused with 400; it now reaches the relay through its entry. Relays listed only in a mux are not searched for other keys, since their entries carry settings meant for that mux's keys; such a relay is dialed like any builder outside the config. Signed block: builder-specs makes JSON the default, and Lighthouse forwards the signed block as JSON, but Commit-Boost answered anything but SSZ with 415, so no builder received a Lighthouse block through it (415 on 17 of 17 blocks on a devnet). The block is forwarded undecoded in the encoding it arrived in, labelled JSON when it is JSON or unlabelled, SSZ when it is SSZ; other types are 415. Own host: on a validator client whose builder URL is Commit-Boost, a key with no builder config sends Commit-Boost's hostname as auth data. When that host resolved to a public address, Commit-Boost dialed it on port 443, where it does not listen; behind a firewall that drops packets, the dial waited out the whole bid timeout. Auth data equal to the host in the request's Host header, in the form of default auth data, is refused with 400 before any lookup. Also: the signed block logs one info line with how many builders accepted it, since a block whose bid came from a dialed builder is accepted by none of the configured ones. The ePBS routes read a builder's error body through one helper, and an empty one on the bid route reads "expected 200 or 204". A dial lookup that fails to run is logged. The cb-pbs testing flag moves to dev-dependencies, so a workspace build never compiles in the switch that turns off the dial check. Each change has a test; others pin the lookup cap holding its slot, a dial never carrying a configured relay's headers, a 204 when no lookup slot is free, and the lookup time taken from the builder's timeout. The docs correct the claims on the signed block, default auth data, routing and dial counting, the lookup time in X-Timeout-Ms, 500 on an internal error, and 555 on get_header_stream.

Brings in 9280d92. Hand resolutions: - execution_payload_bid.rs keeps this branch's rule that a builder's 400 or 401 reaches the beacon node only when the stream has no bid, with #506's log field; - epbs.md keeps the stream troubleshooting rows and drops the two rows #506 removed (the 415 and the "no builder accepted" warning); - metrics-catalog.md keeps this branch's rows, which already cover the stream endpoints. The ePBS stream handshake test now expects the relay's own API key, since #506 gives the mock relays a key distinct from the beacon node's.

A relay with get_header = "stream" now also streams its ePBS bids. Each bid request addressed to it opens ws(s)://<relay>/eth/v1/builder/execution_payload_bid_stream/{slot}/{parent_hash}/{parent_root}/{proposer_pubkey} on the PBS stream reader, with the SignedBuilderRequestAuth as padded standard base64 of its SSZ in X-Request-Auth, and races it against the relay's HTTP request. The bid with the higher value + execution_payment wins, unclamped, and the stream's bid wins a tie. A builder's HTTP 400 or 401 reaches the proposer only when neither leg has a bid. A builder Commit-Boost dials is asked over HTTP only. The stream's bid keeps its SSZ frame, so a beacon node that asks for SSZ gets it unchanged. Each frame is decoded on arrival, so the stream holds only its newest bid. A streamed request can last until the deadline less proposer_deadline_buffer_ms, which #506 requires to be above 0. A failed stream is logged and counted in one place, and the race's docs mark what is unreleased. The stream reports the bid stream series under endpoint="get_execution_payload_bid_stream". The ePBS page gets a Bid streaming section and two troubleshooting rows: a relay that streams get_header but not ePBS bids, and a stream that times out or cannot connect.

Brings in 9280d92. Hand resolutions: - execution_payload_bid.rs keeps this branch's rule that a builder's 400 or 401 reaches the beacon node only when the stream has no bid, with #506's log field; - epbs.md keeps the stream troubleshooting rows and drops the two rows #506 removed (the 415 and the "no builder accepted" warning); - metrics-catalog.md keeps this branch's rows, which already cover the stream endpoints. The ePBS stream handshake test now expects the relay's own API key, since #506 gives the mock relays a key distinct from the beacon node's.

JasonVranek added 7 commits October 5, 2026 21:51

docs(epbs): keep the builder config anchor the troubleshooting table …

ee8b073

…links to

address review comments

588a9a1

test: wait for PBS's startup status check in test_mux to fix race

3375c23

condition and lint test

JasonVranek added this pull request to stack #507 October 7, 2026 04:11

JasonVranek mentioned this pull request Oct 7, 2026

epbs builder api endpoints #505

Merged

JasonVranek added 2 commits October 7, 2026 11:30

normalize gloas version header

ef740dc

Merge branch 'epbs-pr1' into epbs-pr2

92b77ca

JasonVranek requested a review from a team October 7, 2026 22:30

ninaiiad reviewed Oct 8, 2026

View reviewed changes

Comment thread crates/pbs/src/dial.rs Outdated

Base automatically changed from epbs-pr1 to main October 8, 2026 15:02

JasonVranek added 2 commits October 8, 2026 08:18

Merge main after #505

6de1249

This was referenced Oct 9, 2026

ePBS (Gloas) support, part 3: validators' builder config from the Commit-Boost config #508

Closed

ePBS (Gloas) support, part 3: bid streaming #509

Merged

vladimir-ea reviewed Oct 10, 2026

View reviewed changes

Comment thread crates/common/src/pbs/relay.rs

vladimir-ea reviewed Oct 10, 2026

View reviewed changes

Comment thread crates/pbs/src/dial.rs

vladimir-ea reviewed Oct 10, 2026

View reviewed changes

Comment thread crates/pbs/src/dial.rs

JasonVranek added 3 commits October 10, 2026 13:28

JasonVranek requested a review from ManuelBilbao October 12, 2026 19:59

JasonVranek mentioned this pull request Oct 13, 2026

feat(pbs): let a mux override proposer_deadline_buffer_ms #510

Closed

vladimir-ea approved these changes Oct 13, 2026

View reviewed changes

ManuelBilbao reviewed Oct 13, 2026

View reviewed changes

Comment thread crates/pbs/src/dial.rs

Comment thread crates/pbs/src/utils.rs

ManuelBilbao approved these changes Oct 13, 2026

View reviewed changes

JasonVranek force-pushed the epbs-pr2 branch from 9280d92 to 2f24266 Compare October 13, 2026 19:47

JasonVranek merged commit 819cfb5 into main Oct 13, 2026
4 checks passed

JasonVranek deleted the epbs-pr2 branch October 13, 2026 19:48

JasonVranek added a commit that referenced this pull request Oct 13, 2026

Merge main after #506

2c94439

JasonVranek mentioned this pull request Oct 13, 2026

feat(pbs): a switch to turn off dialing, and a 20 ms ePBS buffer default #511

Merged

JasonVranek added a commit that referenced this pull request Oct 14, 2026

Merge main after #506

545903a

Navigation Menu

Repository navigation

feat(pbs): dial builders outside the config for ePBS requests - #506

feat(pbs): dial builders outside the config for ePBS requests#506
JasonVranek merged 15 commits into
mainfrom
epbs-pr2

JasonVranek commented Oct 7, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

JasonVranek commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it does

Scope

Testing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

JasonVranek commented Oct 7, 2026 •

edited

Loading